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©Hybrid Image-Plane/Stereo Manipulation 

This method is robust in the face of calibration errors. 
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A Stereoscopic Pair of Electronic Cameras on a legged mobile exploratory robot acquires images of a 
scene that includes a manipulator arm and a target. The image data are used to guide the end effec- 
tor of the manipulator toward the target. 


Hybrid Image-Plane/Stereo (HIPS) 
manipulation is a method of processing 
image data, and of controlling a robotic 
manipulator arm in response to the 
data, that enables the manipulator arm 
to place an end-effector (an instrument 
or tool) precisely with respect to a target 
(see figure). Unlike other stereoscopic 
machine-vision-based methods of con- 
trolling robots, this method is robust in 
the face of calibration errors and 
changes in calibration during operation. 

In this method, a stereoscopic pair of 
cameras on the robot first acquires im- 
ages of the manipulator at a set of pre- 
defined poses. The image data are 
processed to obtain image-plane coor- 
dinates of known visible features of the 
end-effector. Next, there is computed 
an initial calibration in the form of a 
mapping between (1) the image-plane 
coordinates and (2) the nominal three- 
dimensional coordinates of the noted 
end-effector features in a reference 
frame fixed to the main robot body at 
the base of the manipulator. The nomi- 
nal three-dimensional coordinates are 
obtained by use of the nominal forward 
kinematics of the manipulator arm — 
that is, calculated by use of the cur- 
rently measured manipulator joint an- 
gles and previously measured lengths of 
manipulator arm segments under the 
assumption that the arm segments are 
rigid, that the arm lengths are constant, 
and that there is no backlash. It is un- 


derstood from the outset that these 
nominal three-dimensional coordinates 
are likely to contain possibly significant 
calibration errors, but the effects of the 
errors are progressively reduced, as de- 
scribed next. 

As the end-effector is moved toward 
the target, the calibration is updated 
repeatedly by use of data from newly 
acquired images of the end-effector 
and of the corresponding nominal co- 
ordinates in the manipulator refer- 
ence frame. By use of the updated cal- 
ibration, the coordinates of the target 
are computed in manipulator-refer- 
ence-frame coordinates and then used 
to the necessary manipulator joint an- 
gles to position and orient the end-ef- 
fector at the target with respect to the 


same kinematic model from the cali- 
bration step. As the end-effector/ tar- 
get distance decreases, the computed 
coordinates of the end-effector and 
target become more nearly affected by 
the same errors, so that the differences 
between their coordinates become in- 
creasingly precise. When the end-ef- 
fector reaches the target, the remain- 
ing effective position error is the 
distance that corresponds to more 
than about one pixel in the stereo- 
scopic images of the target. 

This work was done by Eric Baumgartner 
and Matthew Robinson of Caltech for 
NASA’s Jet Propulsion Laboratory. Fur- 
ther information is contained in a TSP (see 
page 1). 
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Partitioning a Gridded Rectangle Into Smaller Rectangles 

A relatively simple algorithm yields nearly square, nearly equally sized segments. 
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A relatively simple algorithm, devised 
for use in an image-data-compression 
application, partitions a rectangular pix- 
elated image (or any other rectangle on 
which a regular rectangular grid has al- 
ready been superimposed) into a speci- 
fied number of smaller rectangles, here- 
after denoted segments. The algorithm 
has the following properties: 


• No floating-point operations are 
needed. 

• The segments tend to be nearly square 
(in the sense that their widths and 
heights in pixel units tend to be nearly 
equal) . 

• The segments tend to have nearly 
equal areas. 

• The algorithm yields valid results (no 


zero-width or zero-height segments) as 
long as the specified number of seg- 
ments, 5, does not exceed the number 
of pixels (equivalently, the number of 
grid cells) . 

The inputs to the algorithm are the 
positive integer 5 plus the positive inte- 
gers h and w, denoting the height and 
width, respectively, of the rectangle in 
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